{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "first_steps_with_tensor_flow.ipynb",
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "provenance": [],
      "collapsed_sections": [
        "ajVM7rkoYXeL",
        "ci1ISxxrZ7v0",
        "copyright-notice"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "source": [
        "#### Copyright 2017 Google LLC."
      ],
      "metadata": {
        "id": "copyright-notice",
        "colab_type": "text"
      },
      "cell_type": "markdown"
    },
    {
      "execution_count": 0,
      "outputs": [],
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "metadata": {
        "cellView": "both",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        },
        "id": "copyright-notice2",
        "colab_type": "code"
      },
      "cell_type": "code"
    },
    {
      "metadata": {
        "id": "4f3CKqFUqL2-",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "cell_type": "markdown",
      "source": [
        " # Premiers pas avec TensorFlow"
      ]
    },
    {
      "metadata": {
        "id": "Bd2Zkk1LE2Zr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " **Objectifs d'apprentissage\u00a0:**\n",
        "  * Concepts fondamentaux de TensorFlow\n",
        "  * Utiliser la classe `LinearRegressor` de TensorFlow pour pr\u00e9dire le prix m\u00e9dian des logements, au niveau des \u00eelots urbains, sur la base d'une seule caract\u00e9ristique d'entr\u00e9e\n",
        "  * \u00c9valuer la justesse des pr\u00e9dictions d'un mod\u00e8le en utilisant la racine carr\u00e9e de l'erreur quadratique moyenne (Root Mean Squared Error, RMSE)\n",
        "  * Am\u00e9liorer la justesse d'un mod\u00e8le en modifiant ses hyperparam\u00e8tres"
      ]
    },
    {
      "metadata": {
        "id": "MxiIKhP4E2Zr",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Les donn\u00e9es sont bas\u00e9es sur le recensement de 1990 de l'\u00c9tat de Californie."
      ]
    },
    {
      "metadata": {
        "id": "6TjLjL9IU80G",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ## Configuration\n",
        "Dans cette premi\u00e8re cellule, vous allez charger les biblioth\u00e8ques n\u00e9cessaires."
      ]
    },
    {
      "metadata": {
        "id": "rVFf5asKE2Zt",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "from __future__ import print_function\n",
        "\n",
        "import math\n",
        "\n",
        "from IPython import display\n",
        "from matplotlib import cm\n",
        "from matplotlib import gridspec\n",
        "from matplotlib import pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn import metrics\n",
        "import tensorflow as tf\n",
        "from tensorflow.python.data import Dataset\n",
        "\n",
        "tf.logging.set_verbosity(tf.logging.ERROR)\n",
        "pd.options.display.max_rows = 10\n",
        "pd.options.display.float_format = '{:.1f}'.format"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ipRyUHjhU80Q",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez ensuite charger votre ensemble de donn\u00e9es."
      ]
    },
    {
      "metadata": {
        "id": "9ivCDWnwE2Zx",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "california_housing_dataframe = pd.read_csv(\"https://download.mlcc.google.com/mledu-datasets/california_housing_train.csv\", sep=\",\")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vVk_qlG6U80j",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez m\u00e9langer les donn\u00e9es de mani\u00e8re al\u00e9atoire, pour \u00eatre s\u00fbr d'\u00e9viter les effets de classement pathologique pouvant nuire aux performances de la descente de gradient stochastique. Vous allez, en outre, mettre \u00e0 l'\u00e9chelle la valeur `median_house_value` pour qu'elle soit exprim\u00e9e en milliers, de sorte que son apprentissage soit un peu plus facile avec des taux dans l'intervalle utilis\u00e9 g\u00e9n\u00e9ralement."
      ]
    },
    {
      "metadata": {
        "id": "r0eVyguIU80m",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "california_housing_dataframe = california_housing_dataframe.reindex(\n",
        "    np.random.permutation(california_housing_dataframe.index))\n",
        "california_housing_dataframe[\"median_house_value\"] /= 1000.0\n",
        "california_housing_dataframe"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HzzlSs3PtTmt",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "-"
        }
      },
      "cell_type": "markdown",
      "source": [
        " ## Analyse des donn\u00e9es\n",
        "\n",
        "Il est conseill\u00e9 de se familiariser avec les donn\u00e9es avant de les exploiter.\n",
        "\n",
        "Nous allons imprimer un bref r\u00e9sum\u00e9 de quelques statistiques utiles sur chaque colonne\u00a0: nombre d'exemples, moyenne, \u00e9cart type, maximum, minimum et divers quantiles."
      ]
    },
    {
      "metadata": {
        "id": "gzb10yoVrydW",
        "colab_type": "code",
        "slideshow": {
          "slide_type": "slide"
        },
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "california_housing_dataframe.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Lr6wYl2bt2Ep",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "-"
        }
      },
      "cell_type": "markdown",
      "source": [
        " ## Construction du premier mod\u00e8le\n",
        "\n",
        "Dans cet exercice, vous allez essayer de pr\u00e9dire la valeur m\u00e9diane d'un logement (`median_house_value`) qui deviendra l'\u00e9tiquette (que l'on d\u00e9signe parfois \u00e9galement sous le nom de cible). Vous utiliserez le nombre de pi\u00e8ces (`total_rooms`) comme caract\u00e9ristique d'entr\u00e9e.\n",
        "\n",
        "**REMARQUE\u00a0:** Les donn\u00e9es se situent au niveau de l'\u00eelot urbain. Cette caract\u00e9ristique repr\u00e9sente donc le nombre total de pi\u00e8ces dans cet \u00eelot.\n",
        "\n",
        "Pour entra\u00eener le mod\u00e8le, vous allez utiliser l'interface [LinearRegressor](https://www.tensorflow.org/api_docs/python/tf/estimator/LinearRegressor) fournie par l'API [Estimator](https://www.tensorflow.org/get_started/estimator) de TensorFlow. Cette API s'occupe d'une bonne partie des m\u00e9canismes sous-jacents du mod\u00e8le de bas niveau, et propose des m\u00e9thodes pratiques pour effectuer les t\u00e2ches d'apprentissage du mod\u00e8le, d'\u00e9valuation et d'inf\u00e9rence."
      ]
    },
    {
      "metadata": {
        "id": "0cpcsieFhsNI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a01\u00a0: D\u00e9finir les caract\u00e9ristiques et configurer les colonnes de caract\u00e9ristiques"
      ]
    },
    {
      "metadata": {
        "id": "EL8-9d4ZJNR7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Pour qu'il soit possible d'importer les donn\u00e9es d'apprentissage dans TensorFlow, vous devez sp\u00e9cifier le type de donn\u00e9es qui se trouve dans chaque caract\u00e9ristique. Vous utiliserez principalement deux types de donn\u00e9es pour cet exercice et les suivants\u00a0:\n",
        "\n",
        "* **Donn\u00e9es cat\u00e9gorielles**\u00a0: il s'agit de donn\u00e9es textuelles. L'ensemble de donn\u00e9es immobili\u00e8res utilis\u00e9 dans cet exercice ne contient aucune caract\u00e9ristique cat\u00e9gorielle. Ce type de donn\u00e9es pourrait \u00eatre le style de logement, le contenu d'une annonce immobili\u00e8re\u2026\n",
        "\n",
        "* **Donn\u00e9es num\u00e9riques**\u00a0: donn\u00e9es repr\u00e9sentant un nombre (entier ou \u00e0 virgule flottante) et que vous souhaitez traiter comme tel. Comme nous le verrons par la suite, vous pouvez, dans certains cas, traiter des donn\u00e9es num\u00e9riques (un code postal, par exemple), comme si elles \u00e9taient de type cat\u00e9goriel.\n",
        "\n",
        "Dans TensorFlow, le type de donn\u00e9es d'une caract\u00e9ristique est indiqu\u00e9 \u00e0 l'aide d'une construction appel\u00e9e **colonne de caract\u00e9ristiques**. Les colonnes de ce type ne stockent qu'une description des donn\u00e9es de la caract\u00e9ristique\u00a0; elles ne contiennent pas les donn\u00e9es proprement dites.\n",
        "\n",
        "Dans un premier temps, vous allez simplement utiliser une caract\u00e9ristique d'entr\u00e9e num\u00e9rique, `total_rooms`. Le code suivant extrait les donn\u00e9es `total_rooms` de l'ensemble `california_housing_dataframe` et d\u00e9finit la colonne de caract\u00e9ristiques \u00e0 l'aide de `numeric_column`, qui pr\u00e9cise que ses donn\u00e9es sont num\u00e9riques\u00a0:"
      ]
    },
    {
      "metadata": {
        "id": "rhEbFCZ86cDZ",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "# Define the input feature: total_rooms.\n",
        "my_feature = california_housing_dataframe[[\"total_rooms\"]]\n",
        "\n",
        "# Configure a numeric feature column for total_rooms.\n",
        "feature_columns = [tf.feature_column.numeric_column(\"total_rooms\")]"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "K_3S8teX7Rd2",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " **REMARQUE\u00a0:** Les donn\u00e9es `total_rooms` se pr\u00e9sentent sous la forme d'un tableau \u00e0 une seule dimension (une liste du nombre total de pi\u00e8ces pour chaque \u00eelot urbain). Il s'agit de la forme par d\u00e9faut pour `numeric_column`. Par cons\u00e9quent, elle ne doit pas \u00eatre transmise en tant qu'argument."
      ]
    },
    {
      "metadata": {
        "id": "UMl3qrU5MGV6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a02\u00a0: D\u00e9finir la cible"
      ]
    },
    {
      "metadata": {
        "id": "cw4nrfcB7kyk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez ensuite d\u00e9finir la cible, \u00e0 savoir la valeur m\u00e9diane d'un logement (`median_house_value`). Ici encore, vous pouvez extraire cette donn\u00e9e de l'ensemble `california_housing_dataframe`\u00a0:"
      ]
    },
    {
      "metadata": {
        "id": "l1NvvNkH8Kbt",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "# Define the label.\n",
        "targets = california_housing_dataframe[\"median_house_value\"]"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4M-rTFHL2UkA",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a03\u00a0: Configurer la classe LinearRegressor"
      ]
    },
    {
      "metadata": {
        "id": "fUfGQUNp7jdL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez ensuite configurer un mod\u00e8le de r\u00e9gression lin\u00e9aire \u00e0 l'aide de LinearRegressor. Vous allez entra\u00eener ce mod\u00e8le \u00e0 l'aide de `GradientDescentOptimizer`, qui met en \u0153uvre une descente de gradient stochastique par mini-lots. L'argument `learning_rate` d\u00e9termine la taille du pas de gradient.\n",
        "\n",
        "**REMARQUE\u00a0:** Par mesure de pr\u00e9caution, un [bornement de la norme du gradient] (https://developers.google.com/machine-learning/glossary/#gradient_clipping) est \u00e9galement appliqu\u00e9 \u00e0 l'optimiseur via `clip_gradients_by_norm`. Le bornement de la norme du gradient permet de s'assurer que la magnitude des gradients reste dans des limites acceptables au cours de l'apprentissage, sans quoi la descente de gradient risque d'\u00e9chouer. "
      ]
    },
    {
      "metadata": {
        "id": "ubhtW-NGU802",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "# Use gradient descent as the optimizer for training the model.\n",
        "my_optimizer=tf.train.GradientDescentOptimizer(learning_rate=0.0000001)\n",
        "my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n",
        "\n",
        "# Configure the linear regression model with our feature columns and optimizer.\n",
        "# Set a learning rate of 0.0000001 for Gradient Descent.\n",
        "linear_regressor = tf.estimator.LinearRegressor(\n",
        "    feature_columns=feature_columns,\n",
        "    optimizer=my_optimizer\n",
        ")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "-0IztwdK2f3F",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a04\u00a0: D\u00e9finir la fonction d'entr\u00e9e"
      ]
    },
    {
      "metadata": {
        "id": "S5M5j6xSCHxx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Pour importer les donn\u00e9es immobili\u00e8res de l'\u00c9tat de Californie dans `LinearRegressor`, une fonction d'entr\u00e9e doit \u00eatre d\u00e9finie. Cette fonction indique non seulement \u00e0 TensorFlow comment effectuer le pr\u00e9traitement des donn\u00e9es, mais aussi comment les mettre en lots, les lire en mode al\u00e9atoire et les r\u00e9p\u00e9ter pendant l'entra\u00eenement du mod\u00e8le.\n",
        "\n",
        "Vous allez, tout d'abord, convertir les donn\u00e9es de la caract\u00e9ristique *pandas* en un dictionnaire de tableaux NumPy. Vous pourrez ensuite utiliser l'[API Dataset] (https://www.tensorflow.org/programmers_guide/datasets) de TensorFlow pour construire un objet d'ensemble de donn\u00e9es \u00e0 partir de ces donn\u00e9es, puis scinder ces derni\u00e8res dans des lots de `batch_size`, de sorte qu'elles soient r\u00e9p\u00e9t\u00e9es pour le nombre indiqu\u00e9 d'it\u00e9rations (num_epochs). \n",
        "\n",
        "**REMARQUE\u00a0:** Lorsque la valeur par d\u00e9faut de `num_epochs=None` est transmise \u00e0 `repeat()`, les donn\u00e9es d'entr\u00e9e sont r\u00e9p\u00e9t\u00e9es ind\u00e9finiment.\n",
        "\n",
        "Ensuite, si `shuffle` est d\u00e9fini sur `True`, les donn\u00e9es seront lues de mani\u00e8re al\u00e9atoire, de fa\u00e7on \u00e0 \u00eatre transmises al\u00e9atoirement au mod\u00e8le au cours de l'apprentissage. L'argument `buffer_size` indique la taille de l'ensemble de donn\u00e9es \u00e0 partir duquel `shuffle` sera \u00e9chantillonn\u00e9 de mani\u00e8re al\u00e9atoire.\n",
        "\n",
        "Pour terminer, la fonction d'entr\u00e9e construit un it\u00e9rateur pour l'ensemble de donn\u00e9es et renvoie le lot de donn\u00e9es suivant \u00e0 LinearRegressor."
      ]
    },
    {
      "metadata": {
        "id": "RKZ9zNcHJtwc",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):\n",
        "    \"\"\"Trains a linear regression model of one feature.\n",
        "  \n",
        "    Args:\n",
        "      features: pandas DataFrame of features\n",
        "      targets: pandas DataFrame of targets\n",
        "      batch_size: Size of batches to be passed to the model\n",
        "      shuffle: True or False. Whether to shuffle the data.\n",
        "      num_epochs: Number of epochs for which data should be repeated. None = repeat indefinitely\n",
        "    Returns:\n",
        "      Tuple of (features, labels) for next data batch\n",
        "    \"\"\"\n",
        "  \n",
        "    # Convert pandas data into a dict of np arrays.\n",
        "    features = {key:np.array(value) for key,value in dict(features).items()}                                           \n",
        " \n",
        "    # Construct a dataset, and configure batching/repeating.\n",
        "    ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit\n",
        "    ds = ds.batch(batch_size).repeat(num_epochs)\n",
        "    \n",
        "    # Shuffle the data, if specified.\n",
        "    if shuffle:\n",
        "      ds = ds.shuffle(buffer_size=10000)\n",
        "    \n",
        "    # Return the next batch of data.\n",
        "    features, labels = ds.make_one_shot_iterator().get_next()\n",
        "    return features, labels"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "wwa6UeA1V5F_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " **REMARQUE\u00a0:** Vous continuerez \u00e0 utiliser cette fonction d'entr\u00e9e dans les prochains exercices. Pour en savoir plus sur les fonctions d'entr\u00e9e et l'API `Dataset`, consultez le [Guide du programmeur de TensorFlow] (en anglais) (https://www.tensorflow.org/programmers_guide/datasets)."
      ]
    },
    {
      "metadata": {
        "id": "4YS50CQb2ooO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a05\u00a0: Entra\u00eener le mod\u00e8le"
      ]
    },
    {
      "metadata": {
        "id": "yP92XkzhU803",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous pouvez maintenant appeler `train()` sur `linear_regressor` pour entra\u00eener le mod\u00e8le. Vous allez encapsuler `my_input_fn` dans un `lambda`\n",
        "afin de pouvoir transmettre `my_feature` et `target` comme arguments (pour en savoir plus, reportez-vous \u00e0 ce [didacticiel sur la fonction d'entr\u00e9e TensorFlow](https://www.tensorflow.org/get_started/input_fn#passing_input_fn_data_to_your_model)). Pour commencer, vous allez effectuer\n",
        "l'apprentissage pour 100\u00a0pas."
      ]
    },
    {
      "metadata": {
        "id": "5M-Kt6w8U803",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "_ = linear_regressor.train(\n",
        "    input_fn = lambda:my_input_fn(my_feature, targets),\n",
        "    steps=100\n",
        ")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7Nwxqxlx2sOv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### \u00c9tape\u00a06\u00a0: \u00c9valuer le mod\u00e8le"
      ]
    },
    {
      "metadata": {
        "id": "KoDaF2dlJQG5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez faire des pr\u00e9dictions sur les donn\u00e9es d'apprentissage afin de d\u00e9terminer dans quelle mesure elles sont adapt\u00e9es \u00e0 votre mod\u00e8le au cours de l'apprentissage.\n",
        "\n",
        "**REMARQUE\u00a0:** L'erreur d'apprentissage mesure \u00e0 quel point votre mod\u00e8le est adapt\u00e9 aux donn\u00e9es d'apprentissage. En revanche, elle ne mesure **_pas_** la qualit\u00e9 de **_g\u00e9n\u00e9ralisation du mod\u00e8le aux nouvelles donn\u00e9es_**. Au cours des prochains exercices, vous examinerez comment scinder les donn\u00e9es afin d'\u00e9valuer la capacit\u00e9 de g\u00e9n\u00e9ralisation de votre mod\u00e8le.\n",
        ""
      ]
    },
    {
      "metadata": {
        "id": "pDIxp6vcU809",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "# Create an input function for predictions.\n",
        "# Note: Since we're making just one prediction for each example, we don't \n",
        "# need to repeat or shuffle the data here.\n",
        "prediction_input_fn =lambda: my_input_fn(my_feature, targets, num_epochs=1, shuffle=False)\n",
        "\n",
        "# Call predict() on the linear_regressor to make predictions.\n",
        "predictions = linear_regressor.predict(input_fn=prediction_input_fn)\n",
        "\n",
        "# Format predictions as a NumPy array, so we can calculate error metrics.\n",
        "predictions = np.array([item['predictions'][0] for item in predictions])\n",
        "\n",
        "# Print Mean Squared Error and Root Mean Squared Error.\n",
        "mean_squared_error = metrics.mean_squared_error(predictions, targets)\n",
        "root_mean_squared_error = math.sqrt(mean_squared_error)\n",
        "print(\"Mean Squared Error (on training data): %0.3f\" % mean_squared_error)\n",
        "print(\"Root Mean Squared Error (on training data): %0.3f\" % root_mean_squared_error)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "AKWstXXPzOVz",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "cell_type": "markdown",
      "source": [
        " S'agit-il d'un bon mod\u00e8le\u00a0? Comment \u00e9valuer l'importance de l'erreur\u00a0?\n",
        "\n",
        "Une erreur quadratique moyenne (MSE) pouvant \u00eatre difficile \u00e0 interpr\u00e9ter, on tient plut\u00f4t compte de la racine carr\u00e9e de l'erreur quadratique\n",
        "moyenne (RMSE). Une propri\u00e9t\u00e9 int\u00e9ressante de RMSE est la possibilit\u00e9 de l'interpr\u00e9ter sur la m\u00eame \u00e9chelle que les cibles d'origine.\n",
        "\n",
        "Comparons la RMSE \u00e0 la diff\u00e9rence des valeurs minimale et maximale de nos cibles\u00a0:"
      ]
    },
    {
      "metadata": {
        "id": "7UwqGbbxP53O",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "min_house_value = california_housing_dataframe[\"median_house_value\"].min()\n",
        "max_house_value = california_housing_dataframe[\"median_house_value\"].max()\n",
        "min_max_difference = max_house_value - min_house_value\n",
        "\n",
        "print(\"Min. Median House Value: %0.3f\" % min_house_value)\n",
        "print(\"Max. Median House Value: %0.3f\" % max_house_value)\n",
        "print(\"Difference between Min. and Max.: %0.3f\" % min_max_difference)\n",
        "print(\"Root Mean Squared Error: %0.3f\" % root_mean_squared_error)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "JigJr0C7Pzit",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " L'erreur couvre pratiquement la moiti\u00e9 de l'intervalle des valeurs cibles. Peut-on faire mieux\u00a0?\n",
        "\n",
        "C'est la question qui agace tous les d\u00e9veloppeurs de mod\u00e8le. \u00c9laborons quelques strat\u00e9gies de base pour r\u00e9duire l'erreur de mod\u00e8le.\n",
        "\n",
        "Pour commencer, vous pouvez examiner dans quelle mesure les pr\u00e9dictions r\u00e9pondent aux cibles qui ont \u00e9t\u00e9 d\u00e9finies, en termes de statistiques r\u00e9capitulatives globales."
      ]
    },
    {
      "metadata": {
        "id": "941nclxbzqGH",
        "colab_type": "code",
        "slideshow": {
          "slide_type": "-"
        },
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "calibration_data = pd.DataFrame()\n",
        "calibration_data[\"predictions\"] = pd.Series(predictions)\n",
        "calibration_data[\"targets\"] = pd.Series(targets)\n",
        "calibration_data.describe()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "E2-bf8Hq36y8",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "-"
        }
      },
      "cell_type": "markdown",
      "source": [
        " Ces informations peuvent s'av\u00e9rer bien utiles. Comment peut-on comparer la valeur moyenne \u00e0 la valeur RMSE du mod\u00e8le\u00a0? Qu'en est-il des divers quantiles\u00a0?\n",
        "\n",
        "Vous pouvez \u00e9galement visualiser les donn\u00e9es et la ligne qui a \u00e9t\u00e9 apprise. Pour rappel, la r\u00e9gression lin\u00e9aire sur une seule caract\u00e9ristique peut \u00eatre repr\u00e9sent\u00e9e par une droite transformant l'entr\u00e9e *x* en la sortie *y*.\n",
        "\n",
        "Vous allez d'abord obtenir un \u00e9chantillon al\u00e9atoire uniforme des donn\u00e9es, de mani\u00e8re \u00e0 cr\u00e9er un diagramme de dispersion lisible."
      ]
    },
    {
      "metadata": {
        "id": "SGRIi3mAU81H",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "sample = california_housing_dataframe.sample(n=300)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "N-JwuJBKU81J",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Vous allez ensuite repr\u00e9senter, sous forme graphique, la ligne qui a \u00e9t\u00e9 apprise, en partant de la pond\u00e9ration de caract\u00e9ristique et du biais du mod\u00e8le, superpos\u00e9e au diagramme de dispersion. La ligne sera affich\u00e9e en rouge."
      ]
    },
    {
      "metadata": {
        "id": "7G12E76-339G",
        "colab_type": "code",
        "slideshow": {
          "slide_type": "-"
        },
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "# Get the min and max total_rooms values.\n",
        "x_0 = sample[\"total_rooms\"].min()\n",
        "x_1 = sample[\"total_rooms\"].max()\n",
        "\n",
        "# Retrieve the final weight and bias generated during training.\n",
        "weight = linear_regressor.get_variable_value('linear/linear_model/total_rooms/weights')[0]\n",
        "bias = linear_regressor.get_variable_value('linear/linear_model/bias_weights')\n",
        "\n",
        "# Get the predicted median_house_values for the min and max total_rooms values.\n",
        "y_0 = weight * x_0 + bias \n",
        "y_1 = weight * x_1 + bias\n",
        "\n",
        "# Plot our regression line from (x_0, y_0) to (x_1, y_1).\n",
        "plt.plot([x_0, x_1], [y_0, y_1], c='r')\n",
        "\n",
        "# Label the graph axes.\n",
        "plt.ylabel(\"median_house_value\")\n",
        "plt.xlabel(\"total_rooms\")\n",
        "\n",
        "# Plot a scatter plot from our data sample.\n",
        "plt.scatter(sample[\"total_rooms\"], sample[\"median_house_value\"])\n",
        "\n",
        "# Display graph.\n",
        "plt.show()"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "t0lRt4USU81L",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Cette ligne initiale semble tr\u00e8s \u00e9loign\u00e9e. Voyez s'il est possible de revenir aux statistiques r\u00e9capitulatives et d'examiner les m\u00eames informations qui y sont encod\u00e9es.\n",
        "\n",
        "Ces \u00e9valuations d'int\u00e9grit\u00e9 laissent supposer qu'il doit \u00eatre possible de trouver une bien meilleure ligne"
      ]
    },
    {
      "metadata": {
        "id": "AZWF67uv0HTG",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "cell_type": "markdown",
      "source": [
        " ## Modifier les hyperparam\u00e8tres du mod\u00e8le\n",
        "Dans le cadre de cet exercice, tout le code ci-dessus a \u00e9t\u00e9 plac\u00e9 dans une seule fonction pour plus de facilit\u00e9. Vous pouvez appeler cette fonction avec diff\u00e9rents param\u00e8tres pour visualiser les effets.\n",
        "\n",
        "Dans cette fonction, vous travaillerez dans 10\u00a0p\u00e9riodes r\u00e9parties uniform\u00e9ment, afin de pouvoir observer l'am\u00e9lioration du mod\u00e8le \u00e0 chaque p\u00e9riode.\n",
        "\n",
        "Pour chaque p\u00e9riode, vous allez calculer et repr\u00e9senter graphiquement la perte d'apprentissage. Cela peut vous aider \u00e0 d\u00e9terminer si un mod\u00e8le a converg\u00e9 ou si des it\u00e9rations suppl\u00e9mentaires sont n\u00e9cessaires.\n",
        "\n",
        "Vous allez \u00e9galement repr\u00e9senter graphiquement les valeurs de biais et de pond\u00e9ration de caract\u00e9ristique apprises par le mod\u00e8le au fil du temps. Il s'agit d'une autre m\u00e9thode pour visualiser la convergence des \u00e9l\u00e9ments."
      ]
    },
    {
      "metadata": {
        "id": "wgSMeD5UU81N",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "def train_model(learning_rate, steps, batch_size, input_feature=\"total_rooms\"):\n",
        "  \"\"\"Trains a linear regression model of one feature.\n",
        "  \n",
        "  Args:\n",
        "    learning_rate: A `float`, the learning rate.\n",
        "    steps: A non-zero `int`, the total number of training steps. A training step\n",
        "      consists of a forward and backward pass using a single batch.\n",
        "    batch_size: A non-zero `int`, the batch size.\n",
        "    input_feature: A `string` specifying a column from `california_housing_dataframe`\n",
        "      to use as input feature.\n",
        "  \"\"\"\n",
        "  \n",
        "  periods = 10\n",
        "  steps_per_period = steps / periods\n",
        "\n",
        "  my_feature = input_feature\n",
        "  my_feature_data = california_housing_dataframe[[my_feature]]\n",
        "  my_label = \"median_house_value\"\n",
        "  targets = california_housing_dataframe[my_label]\n",
        "\n",
        "  # Create feature columns.\n",
        "  feature_columns = [tf.feature_column.numeric_column(my_feature)]\n",
        "  \n",
        "  # Create input functions.\n",
        "  training_input_fn = lambda:my_input_fn(my_feature_data, targets, batch_size=batch_size)\n",
        "  prediction_input_fn = lambda: my_input_fn(my_feature_data, targets, num_epochs=1, shuffle=False)\n",
        "  \n",
        "  # Create a linear regressor object.\n",
        "  my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n",
        "  my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n",
        "  linear_regressor = tf.estimator.LinearRegressor(\n",
        "      feature_columns=feature_columns,\n",
        "      optimizer=my_optimizer\n",
        "  )\n",
        "\n",
        "  # Set up to plot the state of our model's line each period.\n",
        "  plt.figure(figsize=(15, 6))\n",
        "  plt.subplot(1, 2, 1)\n",
        "  plt.title(\"Learned Line by Period\")\n",
        "  plt.ylabel(my_label)\n",
        "  plt.xlabel(my_feature)\n",
        "  sample = california_housing_dataframe.sample(n=300)\n",
        "  plt.scatter(sample[my_feature], sample[my_label])\n",
        "  colors = [cm.coolwarm(x) for x in np.linspace(-1, 1, periods)]\n",
        "\n",
        "  # Train the model, but do so inside a loop so that we can periodically assess\n",
        "  # loss metrics.\n",
        "  print(\"Training model...\")\n",
        "  print(\"RMSE (on training data):\")\n",
        "  root_mean_squared_errors = []\n",
        "  for period in range (0, periods):\n",
        "    # Train the model, starting from the prior state.\n",
        "    linear_regressor.train(\n",
        "        input_fn=training_input_fn,\n",
        "        steps=steps_per_period\n",
        "    )\n",
        "    # Take a break and compute predictions.\n",
        "    predictions = linear_regressor.predict(input_fn=prediction_input_fn)\n",
        "    predictions = np.array([item['predictions'][0] for item in predictions])\n",
        "    \n",
        "    # Compute loss.\n",
        "    root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(predictions, targets))\n",
        "    # Occasionally print the current loss.\n",
        "    print(\"  period %02d : %0.2f\" % (period, root_mean_squared_error))\n",
        "    # Add the loss metrics from this period to our list.\n",
        "    root_mean_squared_errors.append(root_mean_squared_error)\n",
        "    # Finally, track the weights and biases over time.\n",
        "    # Apply some math to ensure that the data and line are plotted neatly.\n",
        "    y_extents = np.array([0, sample[my_label].max()])\n",
        "    \n",
        "    weight = linear_regressor.get_variable_value('linear/linear_model/%s/weights' % input_feature)[0]\n",
        "    bias = linear_regressor.get_variable_value('linear/linear_model/bias_weights')\n",
        "\n",
        "    x_extents = (y_extents - bias) / weight\n",
        "    x_extents = np.maximum(np.minimum(x_extents,\n",
        "                                      sample[my_feature].max()),\n",
        "                           sample[my_feature].min())\n",
        "    y_extents = weight * x_extents + bias\n",
        "    plt.plot(x_extents, y_extents, color=colors[period]) \n",
        "  print(\"Model training finished.\")\n",
        "\n",
        "  # Output a graph of loss metrics over periods.\n",
        "  plt.subplot(1, 2, 2)\n",
        "  plt.ylabel('RMSE')\n",
        "  plt.xlabel('Periods')\n",
        "  plt.title(\"Root Mean Squared Error vs. Periods\")\n",
        "  plt.tight_layout()\n",
        "  plt.plot(root_mean_squared_errors)\n",
        "\n",
        "  # Output a table with calibration data.\n",
        "  calibration_data = pd.DataFrame()\n",
        "  calibration_data[\"predictions\"] = pd.Series(predictions)\n",
        "  calibration_data[\"targets\"] = pd.Series(targets)\n",
        "  display.display(calibration_data.describe())\n",
        "\n",
        "  print(\"Final RMSE (on training data): %0.2f\" % root_mean_squared_error)"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kg8A4ArBU81Q",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ## T\u00e2che\u00a01\u00a0: Obtenir une valeur RMSE inf\u00e9rieure ou \u00e9gale \u00e0 180\n",
        "\n",
        "Modifiez les hyperparam\u00e8tres du mod\u00e8le pour am\u00e9liorer le co\u00fbt et obtenir une meilleure correspondance avec la distribution cible.\n",
        "Si, apr\u00e8s environ cinq minutes, vous ne parvenez toujours pas \u00e0 obtenir une valeur RMSE de 180, v\u00e9rifiez la solution pour afficher une combinaison applicable."
      ]
    },
    {
      "metadata": {
        "id": "UzoZUSdLIolF",
        "colab_type": "code",
        "slideshow": {
          "slide_type": "slide"
        },
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        },
        "cellView": "both"
      },
      "source": [
        "train_model(\n",
        "    learning_rate=0.00001,\n",
        "    steps=100,\n",
        "    batch_size=1\n",
        ")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ajVM7rkoYXeL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher une solution."
      ]
    },
    {
      "metadata": {
        "id": "T3zmldDwYy5c",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "train_model(\n",
        "    learning_rate=0.00002,\n",
        "    steps=500,\n",
        "    batch_size=5\n",
        ")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "M8H0_D4vYa49",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " Il ne s'agit l\u00e0 que d'une configuration parmi d'autres\u00a0; d'autres combinaisons de param\u00e8tres peuvent \u00e9galement donner de bons r\u00e9sultats. Notez que le but de cet exercice n'est pas de trouver le param\u00e8tre *optimal*, mais bien de vous aider \u00e0 percevoir en quoi le r\u00e9glage de la configuration du mod\u00e8le affecte la qualit\u00e9 de pr\u00e9diction."
      ]
    },
    {
      "metadata": {
        "id": "QU5sLyYTqzqL",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "cell_type": "markdown",
      "source": [
        " ### Existe-t-il une m\u00e9thode heuristique standard en mati\u00e8re de r\u00e9glage de mod\u00e8le\u00a0?\n",
        "\n",
        "Il s'agit l\u00e0 d'une question courante. En bref, on peut dire que les effets des diff\u00e9rents hyperparam\u00e8tres d\u00e9pendent des donn\u00e9es. Il n'existe donc pas de r\u00e8gles absolues. Vous devez effectuer des tests sur les donn\u00e9es\u00a0!\n",
        "\n",
        "Cela \u00e9tant dit, voici quelques r\u00e8gles empiriques qui peuvent s'av\u00e9rer utiles\u00a0:\n",
        "\n",
        " * L'erreur d'apprentissage doit diminuer r\u00e9guli\u00e8rement, selon une pente abrupte dans un premier temps, pour finalement se stabiliser \u00e0 mesure que l'apprentissage converge.\n",
        " * Si l'apprentissage n'a pas converg\u00e9, essayez de l'ex\u00e9cuter plus longtemps.\n",
        " * Si l'erreur d'apprentissage diminue trop lentement, augmenter le taux d'apprentissage permettra peut-\u00eatre d'acc\u00e9l\u00e9rer la diminution.\n",
        "   * Cependant, il arrive parfois que l'inverse se produise si le taux d'apprentissage est trop \u00e9lev\u00e9.\n",
        " * Si l'erreur d'apprentissage varie sensiblement, essayez de diminuer le taux d'apprentissage.\n",
        "   * Une bonne m\u00e9thode consiste g\u00e9n\u00e9ralement \u00e0 diminuer le taux d'apprentissage tout en augmentant le nombre de pas ou la taille du lot.\n",
        " * Des lots de tr\u00e8s petite taille peuvent \u00e9galement entra\u00eener une instabilit\u00e9. Commencez par des valeurs telles que 100 ou 1\u00a0000, et continuez \u00e0 r\u00e9duire la taille jusqu'\u00e0 ce que vous constatiez une d\u00e9gradation.\n",
        "\n",
        "Pour rappel, vous ne devez pas suivre \u00e0 la lettre ces r\u00e8gles empiriques, car les effets d\u00e9pendent des donn\u00e9es. Vous devez toujours essayer une m\u00e9thode et v\u00e9rifier ensuite le r\u00e9sultat."
      ]
    },
    {
      "metadata": {
        "id": "GpV-uF_cBCBU",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "cell_type": "markdown",
      "source": [
        " ## T\u00e2che\u00a02\u00a0: Essayer une autre caract\u00e9ristique\n",
        "\n",
        "Voyons s'il est possible d'obtenir un meilleur r\u00e9sultat en rempla\u00e7ant la caract\u00e9ristique `total_rooms` par `population`.\n",
        "\n",
        "Ne consacrez pas plus de cinq minutes \u00e0 cette t\u00e2che."
      ]
    },
    {
      "metadata": {
        "id": "YMyOxzb0ZlAH",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "# YOUR CODE HERE"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ci1ISxxrZ7v0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### Solution\n",
        "\n",
        "Cliquez ci-dessous pour afficher une solution."
      ]
    },
    {
      "metadata": {
        "id": "SjdQQCduZ7BV",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "train_model(\n",
        "    learning_rate=0.00002,\n",
        "    steps=1000,\n",
        "    batch_size=5,\n",
        "    input_feature=\"population\"\n",
        ")"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    }
  ]
}
